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ABSTRACT 


UCAVs are advanced weapon systems that can loiter autonomously in a 
pack over a target area, detect and acquire the targets, and then attack them. 
Modeling these capabilities in a specific hostile operational setting is necessary 
for addressing weapons’ design and operational issues. While much attention 
has been given to the engineering and technological aspects of UCAV 
developments, there are very few studies on operational concepts for these 
weapon systems and their effectiveness and efficiency. This thesis builds 
probability models (Markov Chains) that describe UCAV operations, defines 
Measures of Effectiveness (MOEs) for the engagement performance, maps the 
functional relations between the parameters and the MOEs, and obtains insights 
regarding the design of the UCAVs and their tactical employment. The models 
are used to conduct extensive numerical analysis, based on experimental design 
concepts and traditional sensitivity analysis. The main focus of the analysis is to 
investigate optimal and robust mixes of UCAVs of different types, with respect to 
the MOEs. While in most cases, extreme-point solutions are optimal, there are 
cases where a balanced UCAV mix is better. 
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I. INTRODUCTION 


A. FOREWORD 

In this chapter we present some aspects of Unmanned Aerial Vehicles 
(UAVs) - in particular Unmanned Combat Aerial Vehicles (UCAVs) - and 
describe the problem addressed in this thesis. We also discuss the methods and 
techniques used for solving the problem and for making useful inference from the 
results. Finally, we outline the thesis chapters. 

B. UNMANNED (COMBAT) AERIAL VEHICLES 

UAVs are mobile airborne machines that do not require an on-board 
human operator. Typically they are controlled by a remote operator or 
autonomous control logic (Corner and Lament, 2004). 

The Department of Defense (DoD) defines UAVs as “powered, aerial 
vehicles that do not carry a human operator, use aerodynamic forces to provide 
vehicle lift, can fly autonomously or be piloted remotely, can be expendable or 
recoverable, and can carry a lethal or non-lethal payload.” (Bone and Bolkcom, 
2003). 

UAVs are a critical part of (future) armed forces, that consists of highly 
mobile and network enabled systems with integrated sensors and precision 
munitions. UAVs either provide eyes on the battlefield that trigger the 
deployment of precision munitions by other platforms, or engage targets 
themselves (UCAVs) (Sulewski, 2005). In addition to triggering the deployment 
of precision munitions, and providing situational awareness of the engagement 
area, UAVs assist in all communication aspects throughout the theater of 
operations. 

The increasingly important role of UAVs in warfare is demonstrated by the 
U.S. Army’s resolution to have these systems at the core of its FCS FoS (Future 
Combat Systems Family of Systems). FCS UAVs are broken down into four 
classes according to their capabilities. Class I UAVs provide RSTA 
(Reconnaissance, Surveillance, Target Acquisition) capabilities at the platoon 
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level. Class II UAVs provide RSTA capabilities and target designation at the 
platoon and company level. Class III UAVs provide RSTA capabilities, target 
designation, communication relay, and mine detection at the CAB (Combined 
Arms Battalion) level, while class IV UAVs provide similar capabilities at the UA 
(Unit of Action) level (Sulewski, 2005). 

The future military force will be complex: a highly integrated mix of 
manned and unmanned units. These unmanned units could function individually 
or within a swarm. The addition of unmanned units will decrease the danger that 
soldiers face in direct combat. The tendency is to either have a single operator 
controlling a swarm of UAVs, or to let them operate autonomously with no human 
supervision. The ability to use autonomous vehicles to perform wartime mission 
is an important application in future military operations. Technology in the UAV 
arena is also moving toward smaller and more capable systems. 

One of the initial motivations that served as impetus for developing UAVs 
was that UAVs would be inexpensive. They could be launched into high risk 
missions without risking a costly manned aircraft and the lives of its crew. Of 
course as the UAVs continuously grow in complexity and utility, they also 
increase in cost, and therefore it becomes more crucial for them to be highly 
combat effective (McMindes, 2005). The effectiveness of UAVs in battle 
depends on many factors, some of which are addressed in this thesis. Exploring 
these factors may let us better understand what design characteristics or 
operational decisions would lead to a more effective (and cost-efficient) use of 
UAVs. 

The use of UCAVs removes the risk of aircrew being killed, injured or 
captured if the vehicle is shot down or lost due to mechanical failure. Airframe 
designs can be smaller and lighter than their manned counterparts and can be 
designed for longer endurance. Also, UCAV platforms are cheaper to buy and 
operate, and require less expensive testing and training. These might be among 
the main advantages in future planning (Baggesen, 2005). 

Modern UCAVs are navigated and guided by radar, video, infrared 
cameras, lasers, and Inertial Navigation Systems (INS) and aided by the satellite 
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based Global Positioning System (GPS). The enhancement of sensor systems, 
processor units, decision making algorithms, and terminal seekers leads to 
autonomy for target acquisition, recognition, and attack. These capabilities, 
combined with inexpensive designs and operational opportunities, make UCAVs 
a disruptive technology on the battlefield. UCAVs enable war fighters to attack 
targets with weapon systems that can operate in highly defended areas, and 
cause less collateral damage, due to enhanced precision. There no longer 
seems to be a trade-off between own casualties and the effect of attacks. This is 
especially important for a society that is perceived as being less and less tolerant 
of high-casualty engagements and collateral damage (Baggesen 2005). 

Experts worry that the more abstract the use of weapon systems 
becomes, the more abstract the enemy becomes, and as humans recede from 
the battlefield as combatants, war will become more likely, not less. 

Autonomous technology is still not completely unleashed. For the near 
future, however, UCAV developers believe that the man-in-the-loop will be the 
weakest part of the weapon system because humans will be too slow for the 
decision-making cycle, causing underperformance and collateral damage 
(Baggesen 2005). 


C. PROBLEM STATEMENT 

UCAVs are advanced weapon systems that can loiter autonomously in a 
pack over a target area, detect and acquire the targets, and then engage them. 
Modeling these capabilities in a specific hostile operational setting is necessary 
for addressing weapons’ design and operational issues. 

While much attention is given to the engineering and technological 
aspects of UCAV developments, there are very few studies on operational 
concepts for these weapon systems and their effectiveness and efficiency. The 
wide range of design and operational factors and capabilities of such 
autonomously acting and interacting weapons will most likely lead to a wide 
range of engagement performance in various scenarios. 
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In the present thesis we consider a combat situation involving two types of 
UCAVs against two types of passive ground targets and we seek answers to the 
following issues: 

• The effect of UCAV design and operational parameters on the end state of 
the engagement in the presence of imperfect situational awareness. 

• Suitable Measures Of Effectiveness (MOEs) for measuring the effect of 
UCAV design and tactics. 

• Sensitivity of the values of the MOE’s to the UCAV parameters. 

• Given the capabilities of the various types of UCAVs, the best mix of these 
UCAV’s that optimizes certain operational goals. 

D. TECHNIQUES AND METHODS 

In order to gain insight about real combat situations, we need to first 
model it using appropriate techniques and then analyze it in the hope of deriving 
useful conclusions that will help decision makers make better choices when it 
comes to selecting parameter settings of the weapon systems or developing 
scenarios and deployment tactics. 

For modeling the combat situation addressed in this thesis, a continuous 
time Markov chain (CTMC) model is developed. This model represents all the 
UCAV design and environmental parameters and also contains temporal 
information (i.e., the expected duration of a UCAV operation). The basic idea 
behind the CTMC is that at each moment in time the system can be described by 
the state it is in. The state, in general, contains information about the number of 
UCAVs and targets alive. All of the parameter information is kept in the model 
and is used in order to calculate the next state and the elapsed time. The 
outcomes resulting from the model are used to address the issues described 
above in Section C. In this thesis, two MCEs are calculated, the expected 
relative effectiveness and the expected time of the operation, both described in 
Chapter III. The parameter value settings along with their respective MCE 
outputs are then analyzed according to two techniques. The first one is the 
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NOLH DOE (Nearly Orthogonal Latin Hypercube Design of Experiments). This 
method allows for an exploration as broad as the analyst deems necessary, and 
also allows for exposing potential interactions among the various factors. One 
product of this type of analysis is a regression equation that can be used for its 
explanatory power but also as a quick substitute for the Markov model. The 
second technique is the traditional sensitivity analysis that helps magnify the 
effects of particular factors when everything else is kept constant. These two 
techniques complement each other and provide a comprehensive view of the 
combat situation we are modeling. The statistical package JMP is used for most 
of the DOE and analysis part, and Excel is used for generating and formatting the 
various plots during the analysis. 

E. THESIS FLOW 

In the next chapter we discuss previous UCAV related models that 
employed different approaches and conveyed certain takeaways. Also, we 
expand on the Markov chain and DOE concepts. Chapter III describes the 
combat situation and develops the basic Markov model. It also gives a thorough 
discussion of the analysis performed using various methods, as well as 
conclusions and recommendations. Supporting documentation on the 
programming code is contained in the Appendix. 
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II. BACKGROUND - LITERATURE REVIEW 


In this chapter we discuss previous Operations Research (OR) models 
related to UCAVs, continuous time Markov chains, and basic concepts of 
experimental design related to the analysis presented later on. 

A. PREVIOUS UCAV MODELS 

Progress on the various technologies of UCAVs is promising, but there are 
insufficient analytical tools for evaluating the effectiveness of these weapon 
systems in operational settings. Most of the research on modeling UCAV 
operations relies on simulation and not on analytic modeling. In (Jacques, 2002) 
the author presents some basic analytic results on the single UCAV/single target 
and general multi-UCAV/multi-target cases. It is shown there, that analytically it 
becomes intractable to develop a mathematical formulation for arbitrary numbers 
of munitions executing arbitrarily specified search patterns. These are the cases, 
however, that are most interesting operationally, and the most practical way of 
performing this more general analysis is by a numerical simulation. 

Some work has been done on investigating the possibility of having 
UCAVs share information and act in a cooperative fashion. Cooperative 
behavior is being investigated to improve the overall mission effectiveness. The 
general problem addressed is typically how to best find and engage an unknown 
number of targets in unknown locations using multiple UCAVs. In (Frelinger, et 
al, 1998) it is stated that while an individual UCAV may be less capable than 
conventional munitions, through communication across the swarm of weapons, 
the group may exhibit behaviors and capabilities that can exceed those of more 
conventional systems that do not employ communication between weapons. The 
potential benefits, which come about through shared knowledge include relaxed 
sensor performance requirements, robustness to increases in target location 
errors, and adaptivity to attrition and poor target characterization. In (Gillen and 
Jacques, 2002) an attempt is made to emulate the behavior of UCAVs via 
simulation, and measure their overall expected performance. One extension to 
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the approach described in (Frelinger, et al, 1998) is taking into account the 
degradation due to false target attacks. The simulation allows for any number of 
targets with varying priority levels, as well as non-targets (military or civilian), and 
it is very flexible in its capabilities to handle a multitude of input parameters and 
supply multiple outputs, such as total hits or total kills. During this simulation, the 
UCAVs employ a decision algorithm. It is shown that the selection of the optimal 
weights of the factors in this decision algorithm are very sensitive to almost all 
battlefield characteristics, therefore producing no robust conclusions, apart from 
the fact that cooperative engagement alone is not able to compensate for higher 
false target attack rates. 

In (Kress et al, 2006), several analytic probability models, which range 
from a simple regenerative formula to a large-scale continuous-time Markov 
chain, are developed, with the objective to address design and operational issues 
of UCAVs operating in hostile environments. The focus is on autonomous 
UCAVs, which are designed to operate as a pack of vehicles that autonomously 
search, detect, acquire and attack targets. The main idea is that while target 
detection and recognition capabilities, and weapon accuracy and lethality 
determine the effectiveness of a single UCAV, two phenomena may affect the 
performance of the UCAVs as a pack - multiple acquisitions and multiple kills. 
Also, the impact of memory on the acquisition capabilities of a UCAV is studied in 
this paper. It is shown that, under reasonable assumptions, memory is a rather 
redundant design feature in UCAVs, unless we consider time-critical missions. 
Some other takeaways are that detection rate is a major factor in determining the 
operation length, that attack coordination among UCAVs is not significant (at 
least for certain examined scenarios), and that UCAV sensor specificity is more 
important than sensitivity. The models described in the paper above are limited 
to homogeneous targets and homogeneous UCAVs. The present thesis extends 
these concepts to multi-type UCAVs and non-homogeneous targets. 
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B. CONTINUOUS TIME MARKOV CHAINS 

A Continuous Time Markov Chain (CTMC) is used for modeling a 
probabilistically evolving situation where time is continuous and the time periods 
between changes in the situation follow an exponential distribution. The situation 
is fully described by the state it is in. Each state can transition to another feasible 
state. The selection of the next state is the result of an ‘exponential race’, that is 
the next state is determined by the event that happens first, where the time until 
the next event follows an exponential distribution. Each state can be either 
transient or absorbing. The model keeps on running as long as the visited states 
are transient. When an absorbing state is reached, the process stops. The core 
of our UCAV model is the CTMC ‘engine’, around which other functions can be 
built. The CTMC model calculates the values of the various MOEs, as well as 
the expected duration of the process. The underlying assumption, though, is that 
the rates of events follow a Poisson distribution. This is not a long stretch, if we 
consider that the behavior of many actual systems approximately follows this 
distribution. The exponential assumption, and the consequent ability to use a 
CTMC gives the analyst an enormous analytical and computational advantage 
since the math would be very difficult or intractable otherwise. 

C. ASPECTS OF EXPERIMENTAL DESIGNS AND DATA ANALYSIS 

In this thesis we implement two types of model analysis: an experimental 
design and a single-factor sensitivity analysis with respect to a base case. A 
good Design of Experiments (DOE) allows for simultaneously assessing the 
impact of more than one factor, and identifying potential interactions among the 
factors. On the other hand, single-factor sensitivity analysis, when all other 
factors remain fixed at their base-case values, may better reveal the effect of 
certain parameters in the neighborhood of a realistic base case. 

The primary objectives of computer experiments, according to (Sacks et 
al, 1989) are: predicting the response at untried inputs, optimizing a function of 
the input factors, or calibrating the computer code to physical data. A more 
modern approach (Sanchez, 2001) contends that the appropriate objectives 
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should be: developing a basic understanding of a particular system, finding 
robust decisions, tactics, or strategies, or comparing the merits of various 
decisions. 

There are several DOE structures to choose from. In this thesis we 
employ Latin Hypercubes and more specifically Nearly-Orthogonal Latin 
Hypercubes (NOLH) for various reasons discussed in the next section. 

1. Latin Hypercubes 

The challenge in conducting analysis is in the curse of dimensionality. In 
general we need L x F design points where F is the number of factors and L is 
the number of levels of each factor, in order to cover all the possible 
combinations. This is known as a full factorial design. As we raise the number of 
factors and desired levels to accommodate the idea of data farming the number 
of design points quickly gets out of hand. 

A NOLH DOE addresses how to sample the design space without looking 
at all possible combinations. It is beyond the scope of this thesis to explain in 
detail how and why this works, but we can imagine the NOLH DOE as selecting 
interior points from the parameter space additionally to the corner points that a 
factorial would select. Those interior points are selected such that the correlation 
between factor levels is very low, so that we get a much more complete picture of 
the landscape from which we are sampling. The low correlation and the large 
number of design points allow the analysis of both main effects and interactions 
between factors without sampling at all combinations of levels of each factor. By 
the application of data farming and NOLH a very broad parameter space can be 
explored and robust solutions can be found. A robust solution may not be the 
optimal choice for any given set of parameters, but is a good overall choice given 
a variety of possibilities (McMindes 2005 and Cioppa 2002). 

NOLH is a very good all-purpose design, particularly when all or most of 
the factors are quantitative. It is apparently efficient, it has excellent space-filling 
properties, and it adds flexibility by imposing fewer restrictions on the number of 
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the factors and their levels. Also, it allows us to fit many different types of 
complex metamodels to multiple MOEs. 
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III. THE BASIC MODEL 


A. THE COMBAT SITUATION AND ITS MODELING OBJECTIVES 

1. Detailed Description of the Combat Situation 

The basic combat situation modeled in this thesis is that of a swarm of 
UCAVs loitering over an area of interest, looking for operationally valuable 
ground targets, acquiring them, and finally attacking them. 

There are two types of UCAVs, A and B, and two types of Valuable 
Targets (VTs), 1 and 2, in the target area. Also, there are other types of targets 
of no operational value called type-3 targets. The latter targets, along with all 
killed VTs of any type, are collectively referred to as Non-Valuable Targets 
(NVTs). 

The type-1 and type-2 targets can be anything valuable to the enemy; for 
example type-1 targets could be C2 (Command and Control) vehicles, and type-2 
targets could be soft-skin military trucks. A mix of UCAVs of both types A and B, 
attack the target area. 

Throughout we assume that the total number of attacking UCAVs is 16 
which typically corresponds to two squadrons. The number of targets, of both 
types, varies according to the scenario that is considered. Nonetheless the 
number of targets of each type is confined within a certain range. 

Each UCAV is disposable (as opposed to retrievable), meaning that the 
weapon is an integral part of the aerial vehicle; the UCAV searches, detects, and 
acquires a target, and finally attacks it. Also, once launched, a UCAV cannot 
return to base, so eventually, if not used, it is wasted. 

The UCAVs loiter over the target area and search for targets 
independently, in a random search pattern. We assume that a UCAV has no 
memory of any previous detection, but it has the capability to recognize an NVT, 
although this capability is imperfect; there is some probability for erroneous 
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classifications. Also, there is continuous communication among the UCAVs that 
leads to perfect coordination, in the sense that simultaneous multiple attacks on 
a single target do not occur. 

The targets are passive in the sense that they do not fight back. So, there 
is no UCAV attrition due to fire by the targets. The combat situation we are 
considering has a limited time length which is in all cases small enough 
compared to the fuel capacity of the UCAVs. Therefore, the UCAVs never run 
the risk of being wasted due to fuel shortage. Because there is no reinforcement 
of UCAVs, this combat situation reaches an absorbing state, when all the UCAVs 
have been disposed. 

In summary, each UCAV loiters above the area of interest, until it detects 
a target which it tries to identify. If the target is identified as a NVT, the UCAV 
takes no action and keeps on loitering. If the target is identified as a VT, the 
UCAV promptly acquires and attacks it. The identification may or may not be 
correct. The attack may or may not be successful. In addition to unnecessary or 
unsuccessful attacks, a UCAV can also be wasted due to a mechanical failure. 

2. The Modeling Objectives and MOEs 
a. Objectives 

The combat situation involves several parameters that describe the 
UCAVs’ performance (e.g., kill probabilities), and the operational environment 
(e.g. the number of targets in the area of interest). The values of these 
parameters are not always known accurately and with certainty. For example, 
the mission planners cannot be absolutely positive about the kill probabilities of a 
type-B UCAV, but they may have an idea about the range of possible values for 
those parameters. In the same manner, the planners cannot accurately predict 
the number of type-1 targets the enemy will employ, but they might well be able 
to provide a valid range for it. These issues are discussed in further detail in the 
experimental design sections later on. 

The objective is to build a stochastic model of the combat situation 
described above, and implement it in order to gain insights and produce useful 
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takeaways on design aspects of the UCAVs as well as on operational issues. 
We use two different approaches for the analysis. The first approach is by 
means of a Design Of Experiments (DOE), and the second one is by defining a 
reasonable base case for all the parameter values and performing sensitivity 
analysis for each parameter separately. 

The main focus of the analysis is to investigate optimal UCAV 
mixes with respect to appropriate Measures Of Effectiveness (MOEs) defined 
below, and under uncertainty regarding the values of the parameters. 

b. Measures of Effectiveness 

In order to measure the mission success, we define the following 

MOEs: 

• Expected value of killed targets 

Assuming that type-1 and type-2 targets have operational values assigned 
to them, this is the total accumulated expected value of all the killed 
targets. 

• Expected relative effectiveness 

This is the ratio of the expected total number of killed VTs, over the initial 
total number of VTs. 

• Probability to exceed an operational threshold 

This is the probability that the total number of killed VTs, is at least a given 
percentage of their initial number. 

• Expected time length of the engagement 

This is the expected total duration of the engagement - until all UCAVs 
are disposed. 

The analysis to follow will focus on the first and fourth MOEs. The 
first MOE measures the effectiveness of the UCAVs; the last one measures their 
time-efficiency, which provides insight about the possible success of time critical 
missions. Specifically, it might be the case that an otherwise optimal UCAV 
mission mix, is not optimal when time is of essence. 
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B. DESCRIPTION OF THE MODEL 


1. Markov Chain 

The stochastic model for the combat situation described above is a 
continuous time Markov chain. A step in this chain is defined as either a 
detection event by a UCAV or a UCAV failure. A state in this model is defined by 
the number of UCAVs (of each type) alive and the number of targets (of each 
type) still alive. A transition from one state to the next is a result of one of seven 
possible events: 

• A type-A UCAV kills a type-1 target 

• A Type-A UCAV kills a type-2 target 

• A Type-B UCAV kills a type-1 target 

• A Type-B UCAV kills a type-2 target 

• A Type-A UCAV fails, or misses a VT, or is wasted on a NVT 

• A Type-B UCAV fails, or misses a VT, or is wasted on a NVT 

• A detection is recorded but none of the six events mentioned above 
occurs; the state remains unchanged 

2. Assumptions 

We make the following assumptions; 

a. Imperfect Recognition 

The sensor of each type of UCAV can identify the status of the 
targets (VT or NVT), but the identification is not perfect; it is accurate with a given 
probability, which depends on the UCAV type. 

b. Communication - Coordination and Zero Attack Time 

There is communication among the UCAVs that leads to perfect 
coordination, in the sense that there are no simultaneous multiple attacks on a 
single target. Also, because the attack time is short compared to the detection 
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time and, as mentioned above, there are no simultaneous multiple attacks on the 
same target, we assume that the attack time is zero. 

c. Fixed Total Number of UCA Vs 

The total number of UCAVs is 16. This way we have a convenient 
(and rational) upper limit on the model complexity, and by determining the 
number of type-A UCAVs we also determine the number of type-B ones. 

d. Unlimited Endurance 

UCAVs are assumed to have enough fuel for the purposes of their 
mission, and therefore they can never crash due to fuel shortage. 

e. Disposable UCAVs 

As mentioned in the model description, the UCAVs are disposable, 
and they never run out of fuel. So, even if all the VTs are destroyed, the 
remaining UCAVs, if any left, keep loitering, until they either mistakenly engage 
an NVT or crash due to mechanical failure. This can potentially distort the values 
of our MOE results because the mission duration may be artificially extended, 
after all the targets are killed. But, this could only be a problem in the event 
where there is a significant probability that all the VTs are destroyed. As we see 
later on, this is a very rare event in our analysis. To make this point clearer, 
consider this example: there are only two targets, and presumably six UCAVs are 
sufficient for destroying them all. In that case, if we were to employ more than 
six UCAVs, the duration of the mission would be much longer since the 
redundant UCAVs would loiter until being disposed, and the Ev would be the 
same although at a higher UCAV expense. But again, in the DOEs employed in 
the analysis, this situation is very rare. 

On the other hand, note that, if our definition of absorbing states 
changes to also include the condition ti + t 2 = 0, this potential distortion of MOE 
values mentioned above would never be the case, while at the same time, we 
would no longer have to assume that the UCAVs are disposable. Of course, for 
that to be true, the target recognition has to be perfect. 


17 



f. Passive Targets 

The targets do not shoot back. So, there is no UCAV attrition due to 
enemy fire. This assumption implies that the number of VTs does not affect the 
failure rate of the UCAVs. Nonetheless, we can always incorporate the effects of 
an existing air defense into the aggregated UCAV failure parameters. 

g. Exponential Detection Rate 

The detections follow a Poisson process; therefore the inter¬ 
detection times are exponential random variables, of which we need only define 
the mean. This is a reasonable assumption, because we can think of targets as 
forming a Poisson field. These targets move randomly in the target area, so that 
they are always spatially distributed according to a spatial Poisson distribution. 

h. Exponential Failure Rate 

Mechanical failures follow a Poisson process with a mean that can 
be estimated from statistical data obtained from controlled experiments. In the 
presence of enemy air defense, this failure rate could also incorporate the effects 
of UCAV attrition by the air defense. 

/. Fixed Probabilities 

All the probabilities in the model are fixed numbers that are 
independent of the time and state of the operation. 

J. No Partial Damage 

Damage is not accumulated on a target. If a target has not been 
destroyed during an attack, it is considered as good as new. 

k. Repeated, Random, Independent, Memory-Less Search 

UCAVs search independently, in a random search pattern, and they 
have no memory regarding previous detections. Since we want to model static 
and moving targets, this assumption is necessary, to avoid implementing a 
complex tracking algorithm. 
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3. Notation 

a. Detection Exponential Rates 

K\\ the detection rate of type-i UCAV, i = A, B. 

b. Failure Exponential Rates 

Gi! the failure rate of type-i UCAV, i = A, B. 

c. Specificity 

n: the probability that a type-i UCAV recognizes a NVT as such, i = 
A, B. 

d. Acquisition 

qy: the probability that a type-i UCAV acquires a type-j target, given 
a detection of such a target, i = A, B, and j = 1, 2. 

e. Kill Probabilities 

Py: the probability that a type-i UCAV kills a type-j target, given an 
acquisition of such a target, i = A, B, and j = 1, 2. 

f. Target Values 

Tvi: the military (operational) value of a type-i target, i = 1, 2. 

g. Initial Number of Targets 

Ti! the total initial number of type-i targets, i = 1, 2, 3. 
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h. Initial Number of UCA Vs 

Ni! the initial number of type-i UCAVs, i = A, B. 

/. State Variables 

• ni! the current number of type-i UCAVs, i = A, B. 

• tj! the current number of type-j targets, j = 1, 2. 

4. Mathematical Formulation 
a. Transitions 

Letting the initial state be (Na, Nb, Ti, T 2 ), and the current state (nA, 
hb, ti, t 2 ), we have seen in section 1 above that there are seven possibilities for 
the next state (feasibility permitting). 

The transition probabilities for these seven cases, along with a 
short description of the characteristics of the transitions, are shown below: 

• Cases 1-4: Type-i UCAV kills Type-j target, i = A, B, j = 1, 2. 

‘jPaqi, ^ 

k=l m=l 


• Cases 5 and 6: Type-i UCAV fails, or misses a VT, or is wasted on a NVT, 
i = A, B. 


„ /C) _|_T^ \ ^''k‘lik(l"Pik) 3 2 




k=l k=l 


k=l 


k=l 
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Case 7: State remains unchanged, i = A, B. 





b. Absorbing States 

We define a state as absorbing when nA + ne = 0. Note that, if a 
state with ti + t 2 = 0 is also specified as absorbing, and if n = 1 (i = A, B), we 
could drop the disposable-UCAV assumption. 

c. Feasibility Conditions 

The feasibility condition for the states is: 

NA-nA+NB-nB>Ti-tj+T2-t2 (or equivalently Ti+T 2 -N^-N 3 < tj+t 2 -n^-nB) and all 
‘components’ of each state must be non-negative. Thus we can discard all 
states that do not satisfy this condition. 

d. MOEs 

The two MOEs to be calculated in the analysis are shown below: 

(1) Expected value of killed targets: 

Ev = E[#type-1 targets killed]Tvi + E[#type-2 targets killed]Tv2 

(2) Time (expected length of the engagement) 

Note that both MOEs are derived from the Markov chain model; 
there are no explicit analytical formulas for their calculation. 
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C. IMPLEMENTATION OF THE MODEL 

1. General 

The continuous time Markov chain model and all the subsequent models 
associated with the analysis are implemented using the Matlab programming 
environment. 

2. Matlab Code Objectives 

The code written, accomplishes many tasks: 

• Getting input and assigning values to the parameters 

• Finding and counting the feasible states 

• Distinguishing between transient and absorbing states 

• Mapping and populating the transition probability matrix P 

• Deriving results from Markov chain theory, with respect to P 

• Calculating the MOE values 

• Checking for errors during run-time 

• Generating output 

3. Code 

The Matlab code written for this model implementation appears in the 
Appendix. 

D. BROAD EXPERIMENTAL SET-UP AND DATA ANALYSIS 

We apply a DOE scheme for building a regression meta-model, and 
maximizing the information obtained from a given number of runs of the model. 
The DOE of choice is NOLH for reasons explained in Chapter II. The basic 
concepts and terminology used herein are also presented in Chapter II. 
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1. Factors 

a. Decision Factors 

We are primarily interested in exploring optimal mixes of the two 
types of UCAVs. Since we assume a total number of 16 UCAVs, Nb = 16 - Na, 
and therefore we only have one decision factor, Na. 


b. Environmentai and Design Factors, and Their Ranges 

(1) Environmental (noise) factors 


• Ti e [3, 7] 

• T 2 e [3, 7] 

• Ts e [0, 28] 

• Tvi = 1 

• Tv2e[0, 1] 


[discrete] 

[discrete] 

[discrete] 

[fixed] 

[continuous] 


The absolute expected total value of the engagement is not 
important and therefore it can be scaled such that Tvi is fixed at the value 1, and 
only Tv 2 varies. By doing that, we reduce the dimensionality of the model by one, 
thus making the DOE and the subsequent analysis less cluttered. 










( 2 ) 


qAi e [0.5, 1] 
given a detection 


qA2 


•1b2 


’•1b2 


. 1-2 

given a detection 

•Iai 


gBi e 


’*1ai 


. 1-2 

^iven a detection 
gB 2 e [0.5, 1] 
given a detection 


UCAV-design (noise) factors 

(probability that a type-A UCAV acquires a type-1 target, 
of such a target) 

(probability that a type-A UCAV acquires a type-2 target, 
of such a target) 

(probability that a type-B UCAV acquires a type-1 target, 
of such a target) 

(probability that a type-B UCAV acquires a type-2 target, 
of such a target) 
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• Pa e [0.5, 1.0] (probability that a type-A UCAV recognizes a NVT as 

such) 

• pb e [0.5, 1.0] (probability that a type-B UCAV recognizes a NVT as 

such) 

• pai e [0.4, 1.0] (probability that a type-A UCAV kills a type-1 target, given 
an acquisition of such a target) 

• pa 2 e ^ p,, (probability that a type-A UCAV kills a type-2 target, 

Li.2 “J 

given an acquisition of such a target) 

• pbi e — p*, (probability that a type-B UCAV kills a type-1 target, 

Li.2 

given an acquisition of such a target) 

• Pb 2 e [0.4, 1.0] (probability that a type-B UCAV kills a type-2 target, given 
an acquisition of such a target) 

• Aa e [0.2, 1] (detection rate of type-A UCAV) 

• Ab e (detection rate of type-B UCAV) 

• 0A e [0.005, 0.03] (failure rate of type-A UCAV) 

• 0B e 0 (failure rate of type-B UCAV) 

1.2 J 

Note that some parameters (e.g., qAi and qei) are correlated. The 
reason for doing that is to explore whether reducing the dimensionality of the 
DOE still conveys similar analysis results or not. If so, fewer varying parameters 
will be examined in any consequent analysis thereon, without any loss of 
generality. 

Also note that restricting, for example qei, to be 1 to 1.2 times less 
than qAi, is equivalent to determining that qei is 83% to 100% of qAi. The same 
is true for all the other pairs of correlated parameters. We prefer showing the 1.2 
ratio factor instead of a percentage because in the next section we use this factor 
for narrowing down the DOE (we call it a handicap). 
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Absent hard data, the ranges for the factor values could only be 
educated guesses, based on the existing literature. However, we intentionally 
made these ranges quite broad, because the NOLH DOE (see Ch. II) that we 
employ for the analysis gives us flexibility, which facilitates exploring a broader 
range of factor values. In Section F we set up a narrowed down experimental 
design that significantly reduces the dimensionality of our model, while at the 
same time, maintains the potential for robust inference, and clear conclusions. 

We tried not to give any UCAV a clear overall advantage over the 
other one (otherwise the optimal UCAV mix is trivial). So, a type-A UCAV is 
more effective against a type-1 target, and a type-B UCAV is more effective 
against a type-2 target. Also, a type-A UCAV has a higher detection rate, but is 
more failure-prone than a type-B UCAV. The targets, too, have some 
differences. Type-1 targets are more valuable but harder to acquire than type-2 
targets. 

We seek to identify situations where the mission mix affects the 
outcome of the engagement, despite this balanced setup. 

2. Design of Experiments 

We use a 129-NOLH design for the 18 (varying) noise factors and we 
cross it with the 17 discrete levels of the (unique) decision factor Na. This gives 
129 X 17 = 2,193 design points in total. 

3. Batch-Running the Model 

Since MATLAB® can’t run in command line mode, we cannot use a batch 
file approach to automate the 2,193 runs. Instead, we feed an Excel® file that 
contains all the design points into the MATLAB workspace, and the MATLAB 
code sequentially reads the parameter values corresponding to each design 
point, runs the continuous time Markov model, and generates the output values 
which are then saved to another Excel file. All this input and output (I/O) is 
performed automatically by the MATLAB code without the need for any additional 
setup. 


25 



4. Analysis and Results 

a. Course of Analysis 

The Markov chain model employs various design, operational, and 
decision factors. The relations between these parameters and the MOEs are 
mapped, in order to gain insights regarding the design of the UCAVs and their 
tactical employment. The statistical package JMP® is used to evaluate the MOE 
outputs from the model runs based on sets of inputs that are determined 
according to the NOLH DOE principles. Since we are interested in the optimal 
UCAV mix, we plot the MOE values as functions of Na, in order to observe how 
significant the decision factor is. 

Also we employ regression analysis as our main tool for meta¬ 
modeling the mathematical model. We construct various regression models of 
the MOEs, which include linear, quadratic, and second degree interaction terms, 
for investigating whether a simpler meta-model is more efficient than a more 
accurate one, by being able to capture most of the modeling situation with only a 
subset of the factors. If the regression accounts for most of the variability of our 
model, it can be assigned the role of a ‘hard and fast’ substitute for the model. 
This is convenient when the model is not available, or when the time available for 
running the model is limited. Also, the regression function transparently shows 
which factors have an effect on the MOE, how big the effect is, and in what 
direction. 


b. Analysis 

First we explore how does mission mix (i.e., the Na decision factor) 
affect the average value of the Ey MOE. As we see in Figure 1, although the Ey 
tends to be higher for larger Na values, the effect is insignificant. 

Throughout we follow the convention that plots be accompanied by 

their respective data tables, which in some cases contain also some additional 

information. Flere, in Table 1, the Ev values are shown along with their standard 

deviation values which are fairly constant, ratifying the use of regression 

analysis. Also, we can see the extreme values and the range of Ey values for 

each Na. They all moderately increase with Na, showing that if Na has any effect 
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on Ev it has to be positive, even though the overall effect appears insignificant in 
this case. This insignificance is attributed to the design of the 18-noise-facrtor 
fairly balanced combat situation. 

Note that the plot in Figure 1 is drawn on a magnified scale, and the 
nature of the relation (e.g., quadratic) is revealed, due to the very limited range of 
Ev values; this is indifferent to our conclusions though. In this case, actually, our 
interpretation is that the mission mix makes no practical difference. 


Na 

Mean(Ev) 

StDev(Ev) 

Min(Ev) 

Max(Ev) 

Range(Ev) 

0 

4.112 

1.670 

1.177 

8.958 

7.781 

1 

4.131 

1.643 

1.243 

8.773 

7.530 

2 

4.147 

1.618 

1.308 

8.587 

7.279 

3 

4.163 

1.597 

1.362 

8.399 

7.037 

4 

4.178 

1.580 

1.362 

8.361 

6.999 

5 

4.191 

1.565 

1.362 

8.470 

7.108 

6 

4.203 

1.555 

1.362 

8.576 

7.214 

7 

4.214 

1.548 

1.362 

8.681 

7.319 

8 

4.223 

1.544 

1.362 

8.784 

7.422 

9 

4.232 

1.545 

1.362 

9.061 

7.699 

10 

4.239 

1.549 

1.362 

9.420 

8.058 

11 

4.245 

1.556 

1.362 

9.764 

8.402 

12 

4.250 

1.567 

1.362 

10.088 

8.727 

13 

4.254 

1.581 

1.361 

10.389 

9.028 

14 

4.256 

1.598 

1.361 

10.662 

9.300 

15 

4.257 

1.617 

1.361 

10.904 

9.542 

16 

4.258 

1.638 

1.361 

11.113 

9.752 


Table 1. Statistics on the Ev MOE for the different Na values 



Figure 1. The average Ev MOE values as a function of Na 
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Next, we explore how Na affects the mean time of the operation 
(second MOE). As we can see in Figure 2, the expected time tends to be shorter 
for larger Na values. So, as with Ev, this MOE too, gets a more favorable value 
as Na increases. Note though, that the effect of Na on Time is more significant 
than it is on E^,. 

In Table 2, the Time values are shown along with the respective 
standard deviation values, which are not constant, making the use of regression 
analysis for this MOE less appropriate. Also, we can see the extreme values and 
the range of Time values for each Na. When Na is about its middle value, the 
Time values are more consistent. Because of this behavior, we conclude that 
although the Time is shorter for larger Na, it is wiser to choose an intermediate 
Na value (like 9, in this case), since this gives a worst case (max) Time value 
which is almost half the value we would get if Na was closer to its extreme points 
(0 and 16). 


Na 

Mean(Time) 

StDev(Time) 

Min(Time) 

Max(Time) 

Range(Time) 

0 

22.668 

15.434 

6.623 

117.669 

111.045 

1 

22.581 

14.727 

6.815 

111.031 

104.216 

2 

22.490 

14.073 

7.002 

104.418 

97.416 

3 

22.397 

13.482 

7.184 

97.863 

90.679 

4 

22.301 

12.962 

7.362 

91.403 

84.040 

5 

22.203 

12.524 

7.536 

85.067 

77.530 

6 

22.104 

12.178 

7.590 

78.883 

71.293 

7 

22.005 

11.934 

7.558 

72.875 

65.317 

8 

21.906 

11.800 

7.526 

67.057 

59.531 

9 

21.808 

11.786 

7.493 

61.439 

53.946 

10 

21.711 

11.904 

7.460 

62.108 

54.648 

11 

21.617 

12.167 

7.427 

65.629 

58.202 

12 

21.527 

12.592 

7.394 

69.161 

61.768 

13 

21.439 

13.196 

7.289 

79.896 

72.607 

14 

21.353 

13.997 

7.124 

92.774 

85.650 

15 

21.265 

15.003 

6.956 

107.535 

100.579 

16 

21.170 

16.210 

6.578 

123.934 

117.356 


Table 2. Statistics on the Time MOE for the different Na values 
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Figure 2. The average Time MOE values as a function of Na 


Next we build three regression meta-models of the mathematical 
model. They describe the Ev MOE only (since due to heteroscedasticity we 
deemed Time inappropriate for regression). The first regression only employs 
the (linear) model factors, the second one additionally employs quadratic terms 
(i.e., the model factors squared), and the third one additionally employs all 
possible second degree interactions of the model factors. It is obvious that the 
third meta-model has the most descriptive power, but we explore the other two 
cases because simplicity and fewer terms are desirable regression attributes 
(they aid the interpretation) even when the descriptive power falls a little bit 
behind. 

(1) Regression with linear terms only 

Here, the 18 noise factors and the single decision factor are 
initially added to the regression which sequentially eliminates the factors that are 
not significant. The (as well as the adjusted R^) is about 86%, which is fairly 
high. In Table 3 we see the values of the parameter estimates for this 
regression. Notice how small the standard error for each of the 13 estimates is. 
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making the regression more robust. Also, note that Na is eliminated as a factor 
(i.e., it is statistically insignificant), as we might have expected from the previous 
analysis. 


Response Ev 
Summary of Fit 


RSquare 

0.858 

RSquare Adj 

0.857 

Root Mean Square Error 

0.598 

Mean of Response 

4.209 

Observations (or Sum Wgts) 

2193 


Parameter Estimates 


Term 

Estimate 

Std Error 

t Ratio 

Prob>|t| 

Intercept 

-6.911 

0.171 

-40.44 

<.0001 

Ti 

0.385 

0.009 

42.55 

<.0001 

Tz 

0.104 

0.009 

11.59 

<.0001 

Ta 

-0.075 

0.002 

-47.78 

0.0000 

Tv2 

2.910 

0.044 

66.27 

0.0000 

qAi 

1.245 

0.088 

14.17 

<.0001 

qB2 

0.458 

0.088 

5.22 

<.0001 

Ta 

2.545 

0.088 

28.98 

<.0001 

re 

2.631 

0.088 

29.87 

<.0001 

Pai 

1.325 

0.341 

3.88 

0.0001 

Pbi 

1.618 

0.366 

4.43 

<.0001 

PA2 

1.748 

0.078 

22.30 

<.0001 

Aa 

0.532 

0.055 

9.70 

<.0001 

Oa 

-17.883 

1.909 

-9.37 

<.0001 


Table 3. Regression with linear terms only output 


(2) Regression with quadratic terms also 


This time, the squared factors are also considered for the 
regression. After the elimination process, we are left with 35 estimates. The 


corresponding terms are: 




rT.2 2 2 2 2 

V2 ’ ^ V2 ’^Al ’^Al ’^Bl ’^Bl ’^B2 ’^A2 ’^A2 ’^A ’^A ’ 


1 ?Pb2 ?Pb2 5Pa2 5Pa2 5^A 5^A 5^A ’^A 


n2 

B’^B’ 


The parameter estimates for these terms are shown in Table 4. The R^ value is 
90% in this case. That value is not much larger than 86%, and therefore the use 
of 22 additional terms does not seem to be justified. 

Note how the quadratic terms are centered about the mean value of their 
corresponding linear term. For example, the average Ti value is 5, and therefore 
the quadratic Ti term is (Ti - 5)^ instead of This, according to regression 
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theory, leads to a less biased regression equation. Besides that, note the 
elimination of NAfrom this regression too. 

(3) Regression with all linear, quadratic, and second- 
degree interaction terms 

This regression considers all the factors, their squares, and 
their possible second degree interactions. After the elimination process the 
terms left in the regression are 126 which render the use of this regression 
unreasonable, even though the has climbed up to 98%. 

c. Results 

By employing this broad experimental design setup, it is obvious 
that, due to the balancing of the parameter values, there appears to be no effect 
of Na on Ey. And although there is some effect on the other MOE, Time, it does 
not seem to be that important either. Nevertheless, for Time, we reached the 
important conclusion that absent hard data it is better to employ a balanced mix 
of UCAVs, instead of a biased one where most of the UCAVs are of one type. 

In the next section, we try to decrease the overall noise, by 
employing a different experimental setup. This is based on the current one, but it 
is narrower, by using the ‘handicap’ concept, and selecting discrete values for the 
Tv 2 factor. 

E. NARROWER EXPERIMENTAL SET-UP AND DATA ANALYSIS 

In this DOE, the varying factors are decreased down to 13 as opposed to 
19 factors in the previously discussed DOE. The other six factors are correlated 
to six of the 13 independent factors, according to a handicap that is decided to be 
equal to 1.2. A smaller handicap wouldn’t reveal the various factor effects as 
articulately, whereas a larger one would just provide for the same insight. 
Therefore, we can employ a 1.2 handicap without loss of generality on our 
analysis conclusions. 
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Response Ev 
Summary of Fit 


RSquare 

0.900 

RSquare Adj 

0.898 

Root Mean Square Error 

0.504 

Mean of Response 

4.209 

Observations (or Sum Wgts) 

2193 


Parameter Estimates 


Term 

Estimate 

Std Error 

t Ratio 

Prob>|t| 

Intercept 

-7.324 

0.155 

-47.16 

0.0000 

Ti 

0.384 

0.008 

50.29 

0.0000 

(Ti-5f 

0.032 

0.008 

4.12 

<.0001 

T2 

0.106 

0.008 

13.94 

<.0001 

{J2-5f 

-0.034 

0.009 

-3.89 

0.0001 

Ts 

-0.075 

0.001 

-56.40 

0.0000 

(T3-14.015f 

0.0006 

0.0002 

2.77 

0.0056 

Tv2 

2.951 

0.037 

79.40 

0.0000 

(Tv2-0.500f 

2.398 

0.182 

13.17 

<.0001 

9ai 

0.392 

0.296 

1.33 

0.1847 

(qii-0.750f 

8.661 

1.177 

7.36 

<.0001 

qei 

1.045 

0.316 

3.30 

0.0010 

(qBi-0.684)2 

-4.249 

1.254 

-3.39 

0.0007 

qB2 

0.775 

0.290 

2.68 

0.0075 

9A2 

-0.415 

0.306 

-1.36 

0.1753 

(qA2-0.684)" 

1.788 

0.730 

2.45 

0.0144 

rA 

2.531 

0.074 

34.11 

<.0001 

(rA-0.750)" 

2.541 

0.744 

3.41 

0.0007 

Tb 

2.600 

0.075 

34.74 

<.0001 

(rB-0.750)2 

3.739 

0.702 

5.33 

<.0001 

Pai 

-0.075 

0.331 

-0.23 

0.8207 

(PA1-0.700)" 

6.727 

1.168 

5.76 

<.0001 

Pbi 

3.210 

0.358 

8.97 

<.0001 

(Pbi-0.638)^ 

-15.913 

1.366 

-11.65 

<.0001 

PB2 

1.078 

0.326 

3.30 

0.0010 

(Pb2-0.700)^ 

-2.867 

1.059 

-2.71 

0.0069 

PA2 

0.626 

0.353 

1.77 

0.0761 

(Pa2-0.638)2 

5.258 

1.146 

4.59 

<.0001 

Aa 

-0.192 

0.415 

-0.46 

0.6441 

(Aa-0.600)^ 

2.168 

0.955 

2.27 

0.0234 

Ab 

0.800 

0.455 

1.76 

0.0790 

(Ab-0.547)^ 

-5.114 

1.086 

-4.71 

<.0001 

Oa 

-60.146 

14.240 

-4.22 

<.0001 

(eA-0.017)" 

5483.747 

1331.215 

4.12 

<.0001 

Qb 

47.936 

15.561 

3.08 

0.0021 

(Ob-O.OIG)^ 

-6208.849 

1542.055 

-4.03 

<.0001 


Table 4. Regression with linear and quadratic terms output 
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1. Factors 

a. Decision Factor 

Again, we only have one decision factor, Na, which determines the 

UCAV mix. 

b. Environmentai and Design Factors, and Their Ranges 

(1) Environmental (noise) factors 


• Tie [3,7] [discrete] 

• 12 0 [3,7] [discrete] 

• T 3 e [0,28] [discrete] 

• Tvi = 1 [fixed] 

• Tv 2 e {0,0.5, 1} [discrete] 


Note that when Tv 2 (which is now discrete) is equal to 0, then the 
Tvi/Tv 2 ratio goes to infinity. Killing type-2 targets adds no value to the military 
operation. On the other hand, when Tv 2 is equal to 1, that ratio becomes 1, 
meaning that both types of targets have the same military value and thus their 
value is not a factor mission-wise. 

(2) UCAV-design (noise) factors 

This experimental setup has a reduced dimensionality 
compared to the previous broad setup. For accomplishing this, the correlated 
parameters (e.g., Pai and qei) have a fixed relation, called a handicap, instead of 
a randomly varying relation. So, the handicap is a fixed coefficient by which 
some parameters are inferior compared to their respective correlated 
parameters. In the subsequent analysis, we also explore the effect of different 
handicap values (1.0, 1.1, 1.2) on the inference significance. The default 
handicap is 1.2 (corresponds to 83% inferiority). 

• Pai e [0.5, 1] [continuous] 

• qA 2 = ^ [correlated with qB 2 ] 
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n — 

qBi - — 

[correlated with qAi] 

qB2e[0.5, 1] 

[continuous] 

rAC [0.5, 1.0] 

[continuous] 

re e [0.5, 1.0] 

[continuous] 

Pai e [0.4, 1.0] 

[continuous] 

„ _ Pb2 

O 

[correlated with pb 2 ] 

n - Pai 

■ O 

[correlated with pai] 

Pb 2 e [0.4, 1.0] 

[continuous] 

AAe[0.2, 1] 

[continuous] 

Ab = ^ 

1.2 

[correlated with Aa] 


• 0A e [0.005, 0.03] [continuous] 

• ~ ^ [correlated with Oa] 

This narrower experimental design above, has a significantly 
reduced dimensionality compared to the previous broad DOE, while at the same 
time, as we will see, conveys the same types of results for Ev, when analysis is 
done. For Time, though, we get different results, which now are more consistent 
with the results for Ev. 

Again, the properties of the UCAVs and the targets, are balanced, 
enabling the exploration of non-trivial UCAV mixes. 

2. Design of Experiments 

We use a 33-NOLH design for the 11 (varying) noise factors and we cross 
it with the 3 discrete levels of the Tv 2 noise factor as well as the 17 discrete levels 
of the decision factor Na. This gives 33 x 3 x 17 = 1,683 design points in total, 
30% less than in the initial 129 x 17 DOE, while at the same time we have very 
enhanced resolution (i.e., transparency for the effects of Na and the Tvi/Tv 2 ratio) 
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due to the double crossing in the design. If the design was not crossed, then a 
possible effect on some MOE would not be clearly attributed to either the 
decision factor or a noise factor or the Tvi/Tv 2 ratio, without additional 
investigation. This kind of design leads to clearer cause-and-effect conclusions, 
at the expense of more design points than an equivalent single DOE would have. 

3. Analysis and Results 

a. Course of Analysis 

As in the previous section, we are interested in the optimal UCAV 
mix, and therefore we plot the MOE values as functions of Na, in order to observe 
how significant the decision factor is. We investigate the role of Tv 2 to the 
significance of the Na effect. It turns out that Tv 2 affects only Ev, not Time. The 
results we get are consistent with the previous section analysis results, only more 
pronounced. So, the use of the computationally less expensive narrowed down 
DOE is justified. 

Also we employ regression analysis as our main tool for meta¬ 
modeling the mathematical model. We construct various regression models of 
the MOEs. Again, arguably, the regression function of choice is the regression 
with linear terms only. 

b. Analysis 

Initially, we explore how the mission mix (i.e., the Na decision 
factor) affects the value of the Ev MOE, for various handicap values and Tvi/Tv 2 
ratios. What we first see is that the higher the Tvi/Tv 2 ratio the better the effect of 
Na on Ev shows. Given a high Tvi/Tv 2 ratio, larger handicaps give more 
noticeable effects. Therefore an initial conclusion here is that we should choose 
a large enough handicap for the effects to show, and that when the operational 
values of the two types of targets are close, the effects tend to be masked no 
matter what the handicap is. In the subsequent analysis, whenever a handicap 
has to be employed it will take on a value of 1.2. Note that the selection of a 1.2 
handicap value is arbitrary (for example it could be larger) but without introducing 
any loss of generality. The above conclusions are backed up by the plots in 
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Figures 3 to 5. As always, the plots are accompanied by their respective data 
tables (Table 5 to 7). Note that Figure 3 has Tv 2 = 0, Figure 4 has Tv 2 = 0.5, and 
Figure 5 has Tv 2 = 1 ■ Tvi is fixed to 1 as per the design explained in Section 1 b. 

The conclusion here is that a larger Na is always better for the 
military value of the operation, but we should only strive for it when the 
differences between the two UCAV types are substantial and the operational 
values of the two types of targets are quite distant. 


handicap = 1.1 

Tv2=0 

Na 

Ev 

0 

2.637 

1 

2.660 

2 

2.683 

3 

2.705 

4 

2.725 

5 

2.745 

6 

2.764 

7 

2.782 

8 

2.800 

9 

2.816 

10 

2.832 

11 

2.847 

12 

2.861 

13 

2.874 

14 

2.887 

15 

2.899 

16 

2.911 


handicap = 1.2 

Tv2=0 

Na 

Ev 

0 

2.395 

1 

2.438 

2 

2.478 

3 

2.518 

4 

2.556 

5 

2.593 

6 

2.629 

7 

2.664 

8 

2.697 

9 

2.730 

10 

2.761 

11 

2.791 

12 

2.821 

13 

2.849 

14 

2.877 

15 

2.904 

16 

2.931 


handicap = 1.0 

Tv2=0 

Na 

Ev 

0 

2.910 

1 

2.914 

2 

2.918 

3 

2.921 

4 

2.924 

5 

2.926 

6 

2.927 

7 

2.927 

8 

2.927 

9 

2.925 

10 

2.923 

11 

2.920 

12 

2.916 

13 

2.912 

14 

2.906 

15 

2.900 

16 

2.893 


Table 5. Ey as a function of Na when Tv 2 = 0 for three different handicaps. 
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■ handicap=1.2 ■ handicap=1.1 ^^^^^^*handicap=1.0 


Figure 3. Ev as a function of Na when Tv 2 = 0 for three different handicaps. 


handicap = 1.0 

Tv2=0.5 

Na 

Ev 

0 

4.364 

1 

4.371 

2 

4.377 

3 

4.381 

4 

4.385 

5 

4.388 

6 

4.389 

7 

4.390 

8 

4.389 

9 

4.388 

10 

4.385 

11 

4.381 

12 

4.376 

13 

4.369 

14 

4.362 

15 

4.354 

16 

4.344 


handicap = 1.2 

Tv2=0.5 

Na 

Ev 

0 

3.869 

1 

3.897 

2 

3.924 

3 

3.949 

4 

3.972 

5 

3.993 

6 

4.013 

7 

4.031 

8 

4.047 

9 

4.061 

10 

4.074 

11 

4.085 

12 

4.095 

13 

4.103 

14 

4.109 

15 

4.114 

16 

4.118 


handicap = 1.1 

Tv2=0.5 

Na 

Ev 

0 

4.101 

1 

4.117 

2 

4.132 

3 

4.147 

4 

4.160 

5 

4.171 

6 

4.182 

7 

4.191 

8 

4.199 

9 

4.206 

10 

4.212 

11 

4.216 

12 

4.219 

13 

4.222 

14 

4.223 

15 

4.223 

16 

4.222 


Table 6. Ev as a function of Na when Tv 2 = 0.5 for three different handicaps. 
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“ “ ■ handicap=1.2 ~ handicap=1.1 handicap=1.0 


Figure 4. Ev as a function of Na when Tv 2 = 0.5 for three different handicaps. 


handicap = 1.0 

Tv2=1 

Na 

Ev 

0 

5.819 

1 

5.828 

2 

5.835 

3 

5.841 

4 

5.846 

5 

5.850 

6 

5.852 

7 

5.853 

8 

5.852 

9 

5.850 

10 

5.846 

11 

5.841 

12 

5.835 

13 

5.827 

14 

5.818 

15 

5.807 

16 

5.795 


handicap = 1.2 

Tv2=1 

Na 

Ev 

0 

5.343 

1 

5.357 

2 

5.369 

3 

5.379 

4 

5.387 

5 

5.393 

6 

5.396 

7 

5.397 

8 

5.396 

9 

5.393 

10 

5.387 

11 

5.379 

12 

5.369 

13 

5.356 

14 

5.341 

15 

5.324 

16 

5.305 


handicap = 1.1 

Tv2=1 

Na 

Ev 

0 

5.564 

1 

5.574 

2 

5.582 

3 

5.589 

4 

5.594 

5 

5.597 

6 

5.599 

7 

5.600 

8 

5.599 

9 

5.596 

10 

5.592 

11 

5.586 

12 

5.578 

13 

5.569 

14 

5.558 

15 

5.546 

16 

5.533 


Table 7. Ey as a function of Na when Tv 2 = 1 for three different handicaps. 
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6.0 

5.8 

avg Ev 

5.6 

5.4 

5.2 


“ “ ■ handicap=1.2 ~ handicap=1.1 handicap=1.0 



Figure 5. Ev as a function of Na when Tv 2 = 1 for three different handicaps. 


Adopting a handicap of 1.2, this time we estimate the Ev values 
along with other statistical quantities for different Tv 2 values as a function of Na. 
This data is shown in Tables 9 to 11. By examining the standard deviation 
columns, we observe homoscedasticity and we conclude that a regression would 
be appropriate on Ev as a function of all the model parameters. This is done and 
explained later on in this Section. 


Na 

Mean(Ev) 

StDev(Ev) 

Min(Ev) 

Max(Ev) 

Range(Ev) 

0 

2.395 

1.100 

0.614 

5.614 

5.000 

1 

2.438 

1.091 

0.646 

5.589 

4.943 

2 

2.478 

1.084 

0.678 

5.565 

4.887 

3 

2.518 

1.078 

0.710 

5.541 

4.831 

4 

2.556 

1.074 

0.741 

5.516 

4.775 

5 

2.593 

1.072 

0.771 

5.491 

4.720 

6 

2.629 

1.071 

0.802 

5.467 

4.665 

7 

2.664 

1.071 

0.832 

5.443 

4.611 

8 

2.697 

1.072 

0.861 

5.419 

4.558 

9 

2.730 

1.075 

0.890 

5.395 

4.505 

10 

2.761 

1.078 

0.919 

5.372 

4.453 

11 

2.791 

1.083 

0.947 

5.349 

4.402 

12 

2.821 

1.089 

0.975 

5.326 

4.351 

13 

2.849 

1.096 

1.003 

5.305 

4.302 

14 

2.877 

1.104 

1.030 

5.309 

4.280 

15 

2.904 

1.113 

1.027 

5.413 

4.386 

16 

2.931 

1.122 

1.013 

5.513 

4.500 


Table 8. Statistics on the Ev MOE for the different Na values for Tv 2 = 0. 
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Next, we explore how Na affects the mean time of the operation 
(Time MOE). The trend is the same for different handicaps, therefore we display 
only the findings for a handicap of 1.2. Of course these results are unaffected by 
the Tv 2 value, since it makes no difference on the Time MOE (it only affects Ev). 
Time is consistently more favorable for larger Na values; even the variance gets 
smaller as Na grows. So, for time critical missions, sending as many type-A 
UCAVs as possible, is always a good strategy, no matter how small the 
difference of the two types of UCAVs (up to a reasonable point) or how small the 
difference in the operational values of the targets. 


Na 

Mean(Ev) 

StDev(Ev) 

Min(Ev) 

Max(Ev) 

Range(Ev) 

0 

3.869 

1.305 

1.772 

7.170 

5.398 

1 

3.897 

1.287 

1.918 

7.111 

5.193 

2 

3.924 

1.271 

2.057 

7.050 

4.993 

3 

3.949 

1.258 

2.092 

6.988 

4.896 

4 

3.972 

1.249 

2.098 

6.924 

4.826 

5 

3.993 

1.241 

2.082 

6.858 

4.777 

6 

4.013 

1.237 

2.065 

6.791 

4.726 

7 

4.031 

1.235 

2.048 

6.723 

4.675 

8 

4.047 

1.235 

2.031 

6.653 

4.622 

9 

4.061 

1.239 

2.013 

6.667 

4.653 

10 

4.074 

1.245 

1.995 

6.799 

4.804 

11 

4.085 

1.254 

1.977 

6.929 

4.952 

12 

4.095 

1.266 

1.958 

7.056 

5.098 

13 

4.103 

1.280 

1.939 

7.179 

5.240 

14 

4.109 

1.298 

1.920 

7.299 

5.380 

15 

4.114 

1.317 

1.900 

7.416 

5.516 

16 

4.118 

1.340 

1.879 

7.529 

5.650 


Table 9. Statistics on the Ev MOE for the different Na values for Tv 2 = 0.5. 
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Na 

Mean(Ev) 

StDev(Ev) 

Min(Ev) 

Max(Ev) 

Ranqe(Ev) 

0 

5.343 

1.686 

2.286 

8.726 

6.440 

1 

5.357 

1.659 

2.447 

8.632 

6.185 

2 

5.369 

1.634 

2.601 

8.544 

5.942 

3 

5.379 

1.614 

2.749 

8.575 

5.826 

4 

5.387 

1.596 

2.890 

8.603 

5.713 

5 

5.393 

1.583 

2.937 

8.628 

5.692 

6 

5.396 

1.573 

2.935 

8.651 

5.715 

7 

5.397 

1.567 

2.895 

8.670 

5.775 

8 

5.396 

1.565 

2.855 

8.687 

5.833 

9 

5.393 

1.567 

2.813 

8.701 

5.888 

10 

5.387 

1.573 

2.771 

8.747 

5.976 

11 

5.379 

1.584 

2.728 

8.886 

6.158 

12 

5.369 

1.599 

2.684 

9.023 

6.339 

13 

5.356 

1.619 

2.639 

9.157 

6.518 

14 

5.341 

1.644 

2.593 

9.289 

6.696 

15 

5.324 

1.673 

2.547 

9.419 

6.872 

16 

5.305 

1.707 

2.499 

9.546 

7.047 


Table 10. Statistics on the Ev MOE for the different Na values for Tv 2 


Na 

Mean(Time) 

StDev(Time) 

Min(Time) 

Max(Time) 

Range(Time) 

0 

27.953 

27.697 

8.205 

162.267 

154.062 

1 

27.550 

26.162 

8.062 

153.032 

144.970 

2 

27.133 

24.593 

7.916 

143.425 

135.509 

3 

26.702 

23.005 

7.767 

133.490 

125.722 

4 

26.258 

21.415 

7.615 

123.284 

115.670 

5 

25.803 

19.844 

7.459 

112.885 

105.426 

6 

25.339 

18.322 

7.301 

102.384 

95.083 

7 

24.867 

16.880 

7.139 

91.888 

84.748 

8 

24.391 

15.554 

6.975 

81.516 

74.541 

9 

23.912 

14.383 

6.809 

71.395 

64.586 

10 

23.433 

13.404 

6.640 

61.649 

55.009 

11 

22.956 

12.648 

6.468 

54.610 

48.141 

12 

22.481 

12.133 

6.295 

55.150 

48.854 

13 

22.008 

11.862 

6.121 

55.694 

49.573 

14 

21.534 

11.822 

5.944 

56.241 

50.296 

15 

21.054 

11.994 

5.767 

56.790 

51.023 

16 

20.560 

12.358 

5.588 

57.341 

51.753 


Table 11. Statistics on the Time MOE for the different Na values 
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Figure 6. The average Time MOE values as a function of Na 


Next we build three regression meta-models of the mathematical 
model. They describe the Ev MOE only (since due to heteroscedasticity we 
deem Time inappropriate for regression). The first regression only employs the 
(linear) model factors, the second one additionally employs quadratic terms, and 
the third one additionally employs all possible second degree interactions of the 
model factors. It is obvious that the third meta-model has the most descriptive 
power, but we explore the other two cases because simplicity and fewer terms 
are desirable regression attributes (they aid the interpretation) even when the 
descriptive power falls a little bit behind. 


(1) Regression with linear terms only. 

All 13 factors are employed (none is eliminated). The 
resulting is about 87%. 


Response Ev 
Summary of Fit 


RSquare 

0.874 

RSquare Adj 

0.873 

Root Mean Square Error 

0.613 

Mean of Response 

4.027 

Observations (or Sum Wgts) 

1683 
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Parameter Estimates 


Term 

Estimate 

Std Error 

t Ratio 

Prob>|t| 

Intercept 

-7.069 

0.199 

-35.50 

<.0001 

Ti 

0.341 

0.011 

32.38 

<.0001 

Tz 

0.107 

0.011 

10.09 

<.0001 

Ta 

-0.077 

0.002 

-42.85 

<.0001 

C|A1 

1.548 

0.101 

15.33 

<.0001 

C|B2 

0.591 

0.101 

5.84 

<.0001 

Ta 

2.485 

0.101 

24.59 

<.0001 

re 

2.649 

0.101 

26.15 

<.0001 

Pai 

2.894 

0.084 

34.37 

<.0001 

PB2 

1.373 

0.084 

16.31 

<.0001 

Aa 

0.483 

0.063 

7.67 

<.0001 

Oa 

-14.807 

2.015 

-7.35 

<.0001 

Tv2 

2.685 

0.037 

73.28 

0.0000 

Na 

0.015 

0.003 

5.07 

<.0001 


Table 12. Regression with linear terms only output 

(2) Regression with quadratic terms also 

This time, the squared factors are also considered for the 
regression. After the elimination process, we are left with 22 estimates. The 


corresponding terms are: 


Tl ?Tj jTj jTj Tj ,Ty2 jQaI ’•1b2 Ta Ta ? 

Tb TaI ’PaI ’Pb2 ’Pb2 ’^A A A ’^A ’ 


The R^ is increased 


by about 2% compared to the R^ of the linear regression above. Thus, this 
quadratic regression is deemed inefficient compared to the linear one above. 


Response Ev 
Summary of Fit 


RSquare 

0.896 

RSquare Adj 

0.894 

Root Mean Square Error 

0.559 

Mean of Response 

4.027 

Observations (or Sum Wgts) 

1683 


Parameter Estimates 


Term 

Estimate 

std Error 

t Ratio 

Prob>|t| 

Intercept 

-7.380 

0.194 

-38.03 

<.0001 

Ti 

0.344 

0.009 

35.71 

<.0001 

(Ti-5f 

0.052 

0.012 

4.37 

<.0001 

Tz 

0.109 

0.009 

11.27 

<.0001 

{J2-5f 

0.029 

0.013 

2.26 

0.0240 

Ts 

-0.077 

0.002 

-47.00 

<.0001 

(T3-14.0606f 

0.002 

0.0005 

4.02 

<.0001 

qAi 

1.535 

0.092 

16.68 

<.0001 

9B2 

0.587 

0.092 

6.36 

<.0001 

(qBz-0.7503f 

4.435 

1.306 

3.40 

0.0007 

Ta 

2.479 

0.092 

26.91 

<.0001 
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Term 

Estimate 

Std Error 

t Ratio 

Prob>|t| 

(rA-0.7503)^ 

5.358 

1.026 

5.22 

<.0001 

Tb 

2.656 

0.092 

28.75 

<.0001 

(rB-0.7503f 

12.969 

1.368 

9.48 

<.0001 

Pai 

2.908 

0.077 

37.91 

<.0001 

(pAi-0.7006lf 

-9.674 

0.976 

-9.91 

<.0001 

PB2 

1.362 

0.077 

17.76 

<.0001 

(PB2-0.7006lf 

-4.939 

0.908 

-5.44 

<.0001 

Aa 

0.481 

0.057 

8.38 

<.0001 

Oa 

-14.716 

1.836 

-8.01 

<.0001 

(eA-0.0175f 

-1023.385 

564.542 

-1.81 

0.0700 

Tv2 

2.685 

0.033 

80.43 

0.0000 

Na 

0.015 

0.003 

5.56 

<.0001 


Table 13. Regression with linear and quadratic terms output 


(3) Regression with all linear, quadratic, and second- 
degree interaction terms 

This regression considers all the factors, their squares, and 
their possible second degree interactions. After the elimination process the 
terms left in the regression are 52 which render the use of this regression 
unreasonable, even though the R^ has climbed up to 99%. 


c. Results 

By employing this narrow experimental design setup, we reached 

the following results: 

• A larger Na is always better for the military value of the operation, but it is 
only worth the extra cost when the differences between the two UCAV 
types are substantial and the operational values of the two types of targets 
are quite apart. 

• For time critical missions, sending as many type-A UCAVs as possible, is 
always a good strategy, no matter how small the difference of the two 
types of UCAVs (up to a reasonable point) or how small the difference in 
the operational values of the targets. 

• The regression displayed in Table 12 (linear terms only), is a good quick 
substitute for the model, since it accounts for 87% of the variability, 
without having to run the model. 
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F. BASE CASE AND SENSITIVITY ANALYSIS 

In this chapter we define a base case for our scenario and then explore 
the effect of each parameter on the outcome, when the rest of the parameters 
remain fixed at their base-case values. This type of analysis, which may not 
reveal possible interactions among factors, complements the DOE setup 
described in Sections D and E. It might reveal potential relations that were 
concealed in the DOE analysis, due to averaging and canceling-out phenomena. 

1. Base Case 

In the previous section it was shown that we can get clearer and more 
consistent results if we reduce the dimensionality and variability of the DOE, by 
correlating the six factors qA 2 , Pbi, Pa 2 , Pbi, Ab, and 0 b, to six corresponding 
factors qB 2 , qAi, Pb 2 , Pai, Aa, and 0 a respectively. Therefore, for the sensitivity 
analysis setup, the varying factors are twelve (the other six factors are 
dependent). This time, the independence assumption of the varying factors is 
not required, as was the case with the NOLH DOEs previously discussed. 

a. Decision Variabie 

As in the analyses in the previous sections, we only have one 
decision variable, Na, which determines the UCAV mix. The total number of 
UCAVs is fixed at 16, and in the base case, Na = 8 . 

b. Scenario and Design Factors 

(1) Scenario factors 

• Ti = 5 

• T2 = 5 

• T3 = 10 

• Tvi = 1 

• Tv2 = 0.5 


(2) UCAV-design parameters 




„ _ *^62 
qA2- — 

qe, - — 

C|b2 = 0.8 
Pa = 0.8 
Pb = 0.8 

Pai =0.9 

„ _ Pb2 

PA2 - — 

„ - Pa[ 

PB, - — 

Pb2 = 0.9 
Aa=1 

Ab = ^ 
1.2 

eA = 0.01 

Ob = -^ 
1.2 


[coppelated with qB 2 ] 
[coppelated with qAi] 


[coppelated with pb 2 ] 
[coppelated with pai] 


[coppelated with Aa] 


[coppelated with 0 a] 


c. Results of the Base Case 

Running the model at the base case papametep values, we get 
pesults fop the two MOE values as functions of Na, which ape displayed in Table 
14. These pesults ape gpaphically depicted in Figupe 7 and Figupe 8 fop Ev and 
Time pespectively. We can see that the optimal mix of UCAVs fop the base case 
is Na = 14 and Nb = 3, and that the shoptest Time value occups at Na = 16, so it 
appeaps that a lapgep type-A UCAV ppesence ppovides fop bettep effectiveness 
and time-efficiency. 
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Na 

Ev 

Time 

0 

3.921 

13.940 

1 

4.110 

13.738 

2 

4.284 

13.531 

3 

4.444 

13.322 

4 

4.589 

13.112 

5 

4.718 

12.906 

6 

4.831 

12.706 

7 

4.928 

12.517 

8 

5.009 

12.340 

9 

5.074 

12.179 

10 

5.123 

12.036 

11 

5.156 

11.912 

12 

5.174 

11.810 

13 

5.177 

11.730 

14 

5.166 

11.671 

15 

5.142 

11.634 

16 

5.104 

11.616 


Table 14. Both MOE values for the different Na values. 
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Figure 8. The average Time MOE values as a function of Na 


2. Sensitivity Analysis 

a. Course of Analysis 

Each one of the twelve varying factors - Ti, T2, T3, Tv2, Pm, qB2, rA, 
re, Pai, Pb2, Aa, 0a - consecutively varies within its range, everything else kept 
constant. Values of the MOEs are calculated for all 17 Na values during this 
process, so that plots of the MOEs as functions of the decision factor can be 
plotted for the different levels of the examined scenario or design factor. 

Additionally, we explore the effect on Ev of the T 1 /T 2 ratio and 
handicap ratios on Ey. 

b. Sensitivity Analysis on Ti 

Ti takes on values in the set {3, 5, 7}. The expected value of killed 
targets consistently increases with Ti, at every Na value. The increase rate is 
higher for larger Ti values. The expected operation time decreases as the 
number of type-1 targets increases. This might seem counterintuitive, but in fact 
the more targets there are in the area of interest, the higher the acquisition rate 
of the UCAVs is, and this leads to a decrease in the expected operation time. 
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Note that the combat situation is considered terminated when there are no 
UCAVs left, but there can still be alive targets left in the battlefield. 

The overall conclusion here is that the operational benefit increases 
with the number of type-A UCAVs, but that this effect is even more pronounced 
as Ti increases. So, given a high Ti value, the use of more type-A UCAVs 
becomes more imperative versus the use of type-B ones. Of course, time critical 
missions dictate the use of more type-A UCAVs, no matter how many type-1 
targets there are. 



Figure 9. Ev as a function of Na for different Ti parameter values 
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^ T 1=3 —■— 11=5 — A — T 1=7 


Figure 10. Time as a function of Na for different Ti parameter values 

c. Sensitivity Analysis on T 2 

I 2 takes on values in the set {3, 5, 7}. The plots of Ev and Time as 
a function of I 2 are similar to the previous plots for Ti, and the same conclusions 
naturally hold. The fact that the total Ev value is smaller in the present case is 
attributed to the Tvi base case value being twice that of Tv 2 , and so Ti has a 
greater effect on the Ev maximum value than T 2 does. But, I 2 seems to have a 
greater effect on the Ev increase rate. The conclusion here is that a high Na 
value is always more desirable, even for small T 2 values. 
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Figure 11. Ev as a function of Na for different T 2 parameter values 



Figure 12. Time as a function of Na for different T 2 parameter values 
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d. Sensitivity Anaiysis on T 3 

T 3 takes on values in the set {0, 14, 28}. As the number of active 
NVTs increases, more UCAVs are attrite without adding any operational value to 
the combat mission. The Time MOE is also negatively affected by a larger T 3 
number. Nevertheless, T 3 is an environmental noise factor, on which we can 
exercise no control. The conclusion here is that the mission mix is not a crucial 
factor for the expected military value, but when time is of essence, a larger 
number of type-A targets gives a considerably smaller length of operation. 



Number of type-A UCAVs 


T3=0 —■—13=14 —A—T3=28 


Figure 13. Ey as a function of Na for different T 3 parameter values 



-•—13=0 —■—T3=14 —•—13=28 


Figure 14. Time as a function of Na for different T 3 parameter values 
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e. Sensitivity Analysis on Tv 2 

Tv 2 takes on values in the set {0, 0.5, 1}. Obviously, Tv 2 does not 
affect Time, but only affects Ey. The conclusion is that for Tv 2 values comparable 
or equal to Tvi values, the mission mix is irrelevant, but for comparatively small 
Tv 2 values, a larger number of type-A UCAVs generates a higher operational 
value. 



Figure 15. Ey as a function of Na for different Ty 2 parameter values 



Figure 16. Time as a function of Na for different Ty 2 parameter values 
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f. Sensitivity Analysis on qAi 

qAi takes on values in the set {0, 0.5, 1}. What we see here is that 
the unrealistic qAi value of 0, accounts for totally unfavorable MOE results. 
Nevertheless, cutting the qAi value in half (from its maximum possible value) has 
only minimal effects on both MOE values. A value of 0.5 is already too low for 
realistic situations; therefore the conclusion here is that improving the recognition 
capability of type-A UCAVs should not be a high priority. 



Figure 17. Ev as a function of Na for different qAi parameter values 



Figure 18. Time as a function of Na for different qAi parameter values 
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g. Sensitivity Anaiysis on qei 

qB 2 takes on values in the set {0, 0.5, 1}. When qB 2 varies we 
observe an identical behavior as when qAi varied. Thus, we should not primarily 
spend our resources to improve this capability of type-B UCAVs, either. 



Figure 19. Ev as a function of NAfor different qB 2 parameter values 



Figure 20. Time as a function of Na for different qB 2 parameter values 






h. Sensitivity Anaiysis on rA 

Pa takes on values in the set {0, 0.5, 1}. By observing the following 
two plots we reach many conclusions. Firstly, if the type-A UCAV has perfect 
capability to recognize NVTs, then the more the type-A UCAVs the higher the 
expected operational value. The expected operation time, though, explodes. As 
Pa approaches 0, smaller Na values give a more favorable Ev (and make no 
difference to the (small) expected operation time). If there is no data about the pa 
value, then the best strategy is to employ type-B UCAVs only. 


























/. Sensitivity Anaiysis on rs 

Pb takes on values in the set {0, 0.5, 1}. The results here are 
symmetrical to the results of the previous case. If the type-B UCAV capability to 
recognize NVTs is close to perfect, then the more the type-B UCAVs the higher 
the expected operational value. This, also, happens at a high cost in operational 
time. As Pb approaches 0, larger Na values give a more favorable Ey (and make 
no difference to the (small) expected operation time). If there is no data about 
the pb value, then the best strategy is to employ type-A UCAVs only. 

























j. Sensitivity Analysis on Pai 

Pai takes on values in the set {0, 0.5, 1}. For pai = 1, a larger Na is 
desirable. But the smaller the pai value gets the better we are with a smaller Na. 
Effects on Time are less dramatic than they are on Ev. So, depending on 
whether pai is closer to 1 or to 0, the strategy differs. 















k. Sensitivity Analysis on Pbi 

Pb 2 takes on values in the set {0, 0.5, 1}. The conclusion here is 
that we are always better off with as many type-A UCAVs as possible. 
Nonetheless, for Pb 2 = 1, the mission mix becomes less important. 


Number of type-A UCAV s 


. pB2=0 —pB2=0.5 —A— pB2=1 


Figure 27. Ev as a function of Na for different pb 2 parameter values 


Number of type-A UCAV s 


- pB2=0 —■— pB2=0.5 —A— pB2=1 


Figure 28. Time as a function of Na for different qB 2 parameter values 










/. Sensitivity Anaiysis on Aa 

Aa takes on values in the set {0.5, 1, 2}. The plots for this factor 
show that Ev is not significantly affected by Aa, but Time is. We conclude that it is 
the magnitude of Aa (and therefore of Ab too) that determines the best mission 
mix time-wise. When Aa is large enough, we should strive for more type-A 
UCAVs, otherwise the opposite is true. 



Figure 29. Ey as a function of Na for different Aa parameter values 



Figure 30. Time as a function of Na for different Aa parameter values 
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m. Sensitivity Anaiysis on 6 a 

0A takes on values in the set {0.005, 0.010, 0.020}. The effect of Oa 
on Ev is minimal. The same is true for Time. Note that if the operation time is a 
concern, then the best strategy remains to employ as many type-A UCAVs as 
possible, regardless of the Oa value. 



Figure 31. Ev as a function of Na for different Oa parameter values 



Figure 32. Time as a function of Na for different Oa parameter values 










n. Additional Exploration 

In this section, we explore the effect that the T 1 /T 2 ratio and the 
handicap ratio have on the Ev value. This is one example of the many uses of 
our model. It demonstrates that by simultaneously varying more than one factor 
(and more so ratios of factors) we can gain additional insight, due to interactions 
revealed. This example lies between the NOLH DOE and the strict one¬ 
dimensional sensitivity analysis approaches. 

The three figures below expose the main concept: if the ratio T 1 /T 2 
is equal to 1, then the optimal UCAV mix balances in the middle (i.e., Na = 8). 
When T 1 /T 2 is greater than 1, then the optimal mix tends to be displaced to the 
right (i.e., Na > 8), and when the ratio is smaller than 1, the best mission mix is 
displaced to the left (i.e., Na < 8). The further the T 1 /T 2 ratio is from 1, the further 
the optimal Na value is from the middle value (i.e., 8). These effects are more 
dramatic, for larger handicaps, as is displayed by the three plots in each figure. 


7.0 

6.5 

6.0 

Ev 5.5 

5.0 

4.5 

4.0 

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 

Number of type-A UCAV's 

I ■pA1=pB2=Q.8,pA2=pB1=0.2 ^all pij’s =0.5 ■ - ■ pA1=pB2=1, pA2=pB1=Q | 

Figure 33. Ev as a function of Na for different py parameter scenarios when 

Ti=4 and T2=4 


T1=T2=4 
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Number of type-A UCAV's 






















G. CONCLUSIONS 

As is often the case with DoD exploratory analysis, the model has very 
limited predictive power. Indeed, due to a lack of data, the model cannot be 
empirically validated. Rather, the model is used in a descriptive mode to help us 
devise new ideas or assess the consequences of certain assumptions. Potential 
insights gleaned from such exploration usually need to be tested elsewhere, 
perhaps by field experiments. 

In this exploratory analysis we are primarily trying to identify the optimal 
mission mix, and secondarily the factors that have a strong effect on the MOE 
values, the directions of those effects, and which, if any, factors interact. 

By employing the broad experimental design setup, we observe that Na 
( mission mix) does not significantly affect Ev (expected military value). This 
observation is due to the balancing effect of the factor values. Although there is 
some effect on the other MOE, Time, it does not seem to be that significant 
either. Nevertheless, for a more robust Time outcome, absent hard data, it is 
better to employ a balanced mix of UCAVs of different types, instead of a biased 
one where most of the UCAVs are of the same type. 

By employing the narrow experimental design setup, we decreased the 
overall design noise, and we reached the following results: 

• A larger value of Na is always better for the military value of the operation, 
but it is only worth the extra cost (note that a type-A UCAV is more costly 
than a type-B UCAV) when the differences in design characteristics (like 
kill probability, detection rate, etc) between the two UCAV types are 
substantial and the operational values of the two types of targets are quite 
apart. 

• For time critical missions, sending as many type-A UCAVs as possible, is 
always a good strategy, no matter how small the difference between the 
two types of UCAVs (note that if there are no differences then type-B 
UCAVs should be employed exclusively since they are less expensive) or 
how small the difference in the operational values of the targets. 
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By running a sensitivity analysis on our model, we reach some 

conclusions pertaining to the individual parameters as follows: 

• Given a high Ti value (number of type-1 targets), the use of more type-A 
UCAVs becomes more imperative versus the use of type-B ones. Of 
course, time critical missions dictate the use of more type-A UCAVs, no 
matter how many type-1 targets there are. 

• A higher Na value is always more desirable, even for small T 2 values 
(number of type-2 targets). 

• For Tv 2 values (type-2 target military value) comparable to Tvi values 
(type-1 target military value), the mission mix is irrelevant, but for 
comparatively small Tv 2 values, a larger number of type-A UCAVs 
generates a higher operational value. 

• Improving the recognition capabilities (which are typically good enough 
already) of UCAVs (of either type) should not be on a high priority (i.e., we 
should allocate available resources into improving other aspects first). 

• If there is no data about the recognition capabilities of a type-A UCAV (rA), 
then the best strategy is to employ type-B UCAVs only. Otherwise, if the 
type-A UCAV capability to recognize NVTs is close to perfect, then the 
more the type-A UCAVs the higher the expected operational value. In this 
case, though, the expected operation time gets long. As rA approaches 0, 
smaller Na values give a more favorable Ev (and make no difference to the 
(small) expected operation time). 

• The conclusions for re (recognition capability of type-B UCAV) are 
symmetrical to the conclusions for rA stated above. 

• For high pai (kill probability of type-A UCAV against a type-1 target) values 
(i.e., close to 1), a larger Na is desirable. The smaller the pai value gets 
the better we are with a smaller Na. Effects on Time are less dramatic 
than they are on Ey. So, depending on whether pai is closer to 1 or to 0, 
the strategy differs. 
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• No matter what the value of Pb 2 (kill probability of type-B UCAV against a 
type-2 target) is, we are always better off with as many type-A UCAVs as 
possible. Nonetheless, for very high pb 2 values the mission mix becomes 
less important. Note that, as it is implied by the experimental design, the 
Pb 2 value is correlated (i.e., not far from) to the pa 2 value, for this 
conclusion to hold. 

• Ev is not significantly affected by Aa (type-A UCAV detection rate), but 
Time is, and it is the relationship between Aa and Ab (type-B UCAV 
detection rate) that determines the best mission mix time-wise. When Aa 
is greater than Ab, we should strive for more type-A UCAVs and when Ab is 
greater than Aa the opposite is true. 

• The effect of 0 a (type-A UCAV failure rate) on Ev is minimal. The same is 

X 

true for Time. Of course this is due to the large — ratio (by the UCAV 

0A 

design). 
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APPENDIX. MATLAB CODE FOR THE BASIC MODEL 


This Appendix contains the Matlab code for the basic model. By adopting 
this code and slightly modifying it, we can also implement the extension of the 
basic model presented in Chapter IV, or even other potential extensions. 


format compact 

format short 

A = xlsread('DOEinput.xls'); 
[row, col] = size(A); 

N = 16; %hard-wired value 

u = zeros(2); 
u(1,1) = 0; 
u(1,2) = 0; 
u(2,1) = 0; 
u(2,2) = 0; 

for] = 1:row 

T1 = AG,1); %input 
T2 = AG,2); %input 
T3 = AG,3); %input 

Tvalue = zeros(2,1); 
Tvalue(l) = AG,4); %input 
Tvalue(2) = AG,5); %input 

q = zeros(2); 
q(1,1) = AG,6); %input 
q(2,1) = AG,7); %input 
q(2,2) = AG,8); %input 
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q(1,2) = AG,9); %input 


r = zeros(2,1); 
r(1) = AG,10); %input 
r(2) = AG,11); %input 

p = zeros(2); 
p(1,1) = AG,12); %input 
p(2,1) = AG,13); %input 
p(2,2) = AG,14); %input 
p(1,2) = AG,15); %input 

lamda = zeros(2,1); 
lamda(1) = AG,16); %input 
lamda(2) = AG,17); %input 

theta = zeros(2,1); 
theta(1) = AG,18); %input 
theta(2) = AG, 19); %input 

N1 = AG,20); %input 
N2 = N-N1; %derived 

n1 = 0; 
n2 = 0; 
t1 = 0; 
t2 = 0; 


numFeas = 0; %counts total number of feasible states 
numAbs = 0; %counts total number of absorbing states 

rn = (N1 + 1)*(N2 + 1)*(T1 + 1)*(T2 + 1); 

state = zeros(rn, 6); 

for n1 = 0:N1 
for n2 = 0:N2 
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fort1 = 0:T1 
for t2 = 0:T2 

if(T1+T2-N1-N2<=t1+t2-n1-n2 & n1>=0 & n2>=0 & t1>=0 & t2>=0) 
numFeas = numFeas + 1; 
state(numFeas, 1) = numFeas; 
state(numFeas, 2) = n1; 
state(numFeas, 3) = n2; 
state(numFeas, 4) = t1; 
state(numFeas, 5) = t2; 
state(numFeas, 6) = 0; 
if( (n1 +n2) == 0 ) 

numAbs = numAbs + 1; 

state(numFeas, 6) = 1; %this flags an absorbing state 
end 
end 
end 
end 
end 
end 

stateAbs = zeros(numAbs, 6); %temp storage of absorbing states 
numTrans = numFeas - numAbs; %number of transient states 
stateTrans = zeros(numTrans, 6); %temp storage of transient states 
nextAbs = 1; %counter for the next available line of 'stateAbs' 
next!rans = 1; %counter for the next available line of 'stateTrans' 

for ix = 1 mumFeas 
if( state(ix,6) == 1 ) 
stateAbs(nextAbs, :) = state(ix, :); 

stateAbs(nextAbs, 1) = nextAbs; %allowing for consecutive indices 
nextAbs = nextAbs + 1; 
else 

stateTrans(nextTrans, :) = state(ix, :); 
stateTrans(nextTrans, 1) = numAbs + nextTrans; 
nextTrans = nextTrans + 1; 
end 
end 
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state = [stateAbs ; stateTrans]; 


state = state(1 :numFeas, :); 

P = zeros(numFeas); 

for ix = 1 :numFeas 
for iy = 1:numFeas 

%Case 1: from (n1, n2, t1, t2) to (n1 -1, n2, t1 -1, t2) state 
if( state(ix,2)-1 == state(iy,2) & state(ix,3) == state(iy,3)... 

& state(ix,4)-1 == state(iy,4) & state(ix,5) == state(iy,5)) 
P(ix,iy) = 1; 

%Case 2: from (n1, n2, t1, t2) to (n1 - 1, n2, t1, t2) state 
elseif ((state(ix,2)-1 == state(iy,2)) & (state(ix,3) == ... 
state(iy,3)) & (state(ix,4) == state(iy,4)) & ... 

(state(ix,5) == state(iy,5))) 

P(ix,iy) = 2; 

%Case 3: from (n1, n2, t1, t2) to (n1, n2 -1, t1, t2) state 
elseif ((state(ix,2) == state(iy,2)) & (state(ix,3)-1 == ... 
state(iy,3)) & (state(ix,4) == state(iy,4)) & ... 

(state(ix,5) == state(iy,5))) 

P(ix,iy) = 3; 

%Case 4: from (n1, n2, t1, t2) to (n1 -1, n2, t1, t2 -1) state 
elseif ((state(ix,2)-1 == state(iy,2)) & (state(ix,3) == ... 
state(iy,3)) & (state(ix,4) == state(iy,4)) & ... 

(state(ix,5)-1 == state(iy,5))) 

P(ix,iy) = 4; 

%Case 5: from (n1, n2, t1, t2) to (n1, n2 -1, t1 -1, t2) state 
elseif ((state(ix,2) == state(iy,2)) & (state(ix,3)-1 == ... 
state(iy,3)) & (state(ix,4)-1 == state(iy,4)) & ... 

(state(ix,5) == state(iy,5))) 

P(ix,iy) = 5; 

%Case 6: from (n1, n2, t1, t2) to (n1, n2 -1, t1, t2 -1) state 
elseif ((state(ix,2) == state(iy,2)) & (state(ix,3)-1 == ... 
state(iy,3)) & (state(ix,4) == state(iy,4)) & ... 

(state(ix,5)-1 == state(iy,5))) 

P(ix,iy) = 6; 
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%Case 7: from (n1, n2, t1, t2) to (n1, n2, t1, t2) state (the same) 
elseif ((state(ix,2) == state(iy,2)) & (state(ix,3) == ... 
state(iy,3)) & (state(ix,4) == state(iy,4)) & ... 

(state(ix,5) == state(iy,5))) 

P(ix,iy) = 7; 
else 

%None of the 7 cases; meaning that no transition happens 
P(ix,iy) = 0; 
end 
end 
end 

[x1,y1] = find(P == 1); 

[x2,y2] = find(P == 2); 

[x3,y3] = find(P == 3); 

[x4,y4] = find(P == 4); 

[x5,y5] = find(P == 5); 

[x6,y6] = find(P == 6); 

[x7,y7] = find(P == 7); 

%Case 1 

Ien1 = Iength(x1); 

for ix = 1 :len1 

P(x1(ix),y1(ix)) = Iamda(1)*p(1,1)*state(x1(ix),2)*state(x1(ix),4)/.. 
((Iamda(1)+theta(1))*state(x1(ix),2) + (lamda(2)+theta(2))... 
*state(x1(ix),3))/(T1+T2+T3)*(q(1,1)+u(1,2)*(1-q(1,1))); 
end 

%Case 2 
Ien2 = Iength(x2); 
for ix = 1 :len2 

P(x2(ix),y2(ix)) = state(x2(ix),2)*theta(1)/((lamda(1)+theta(1)) ... 
*state(x2(ix),2) + (Iamda(2)+theta(2))*state(x2(ix),3)) + ... 
state(x2(ix),2)*lamda(1 )/((lamda(1 )+theta(1 ))*state(x2(ix),2)+ .. 
(lamda(2)+theta(2))*state(x2(ix),3))/(T1+T2+T3)*... 
(state(x2(ix),4)*(q(1,1 )+u(1,2)*(1-q(1,1 )))*(1-p(1,1))+ ... 
state(x2(ix),5)*(q(1,2)+u(1,1 )*(1-q(1,2)))*(1-p(1,2))+ ... 
(T1+T2-state(x2(ix),4)-state(x2(ix),5)+T3)*(1-r(1))); 
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end 

%Case 3 
Ien3 = Iength(x3); 
for ix = 1 :len3 

P(x3(ix),y3(ix)) = state(x3(ix),3)*theta(2)/((lamda(1)+theta(1)) ... 
*state(x3(ix),2) + (Iamda(2)+theta(2))*state(x3(ix),3)) + ... 
state(x3(ix),3)*lamda(2)/((lamda(1)+theta(1))*state(x3(ix),2)+ 
(lamda(2)+theta(2))*state(x3(ix),3))/(T1+T2+T3)*... 
(state(x3(ix),4)*(q(2,1)+u(2,2)*(1-q(2,1)))*(1-p(2,1))+ ... 
state(x3(ix),5)*(q(2,2)+u(2,1)*(1-q(2,2)))*(1-p(2,2))+ ... 
(T1+T2-state(x3(ix),4)-state(x3(ix),5)+T3)*(1-r(2))); 
end 

%Case 4 
Ien4 = Iength(x4); 
for ix = 1 :len4 

P(x4(ix),y4(ix)) = lamda(1 )*p(1,2)*state(x4(ix),2)*state(x4(ix),5)/ 
((Iamda(1)+theta(1))*state(x4(ix),2) + (lamda(2)+theta(2))... 
*state(x4(ix),3))/(T1+T2+T3)*(q(1,2)+u(1,1)*(1-q(1,2))); 
end 

%Case 5 
Ien5 = Iength(x5); 
for ix = 1 :len5 

P(x5(ix),y5(ix)) = lamda(2)*p(2,1 )*state(x5(ix),3)*state(x5(ix),4)/ 
((Iamda(1)+theta(1))*state(x5(ix),2) + (lamda(2)+theta(2))... 
*state(x5(ix),3))/(T1+T2+T3)*(q(2,1)+u(2,2)*(1-q(2,1))); 
end 

%Case 6 
Ien6 = Iength(x6); 
for ix = 1 :len6 

P(x6(ix),y6(ix)) = Iamda(2)*p(2,2)*state(x6(ix),3)*state(x6(ix),5)/ 
((Iamda(1)+theta(1))*state(x6(ix),2) + (lamda(2)+theta(2))... 
*state(x6(ix),3))/(T1+T2+T3)*(q(2,2)+u(2,1)*(1-q(2,2))); 
end 

%Case 7 
Ien7 = Iength(x7); 
for ix = 1 :len7 

%Distinguishing between absorbing and non-absorbing states 
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if (state(x7(ix),2) == 0 & state(x7(ix),3) == 0) 

P(x7(ix),y7(ix)) = 1; 
else 

P(x7(ix),y7(ix)) = Iamda(1)*state(x7(ix),2)/... 

((Iamda(1)+theta(1))*state(x7(ix),2) + (lamda(2)+theta(2)) *... 
state(x7(ix),3))/(T1+T2+T3)*(state(x7(ix),4)*... 
(1-u(1,2))*(1-q(1,1))+state(x7(ix),5)*(1-u{1,1))*... 
(1-q(1,2))+(T1+T2-state(x7(ix),4)-state(x7(ix),5)+T3)*r(1))... 

+ Iamda(2)*state(x7(ix),3)/... 

((Iamda(1)+theta(1))*state(x7(ix),2) + ... 

(lamda(2)+theta(2))*state(x7(ix),3))/(T1+T2+T3)*(state(x7(ix),4)*... 
(1-u(2,2))*(1-q(2,1))+state(x7(ix),5)*(1-u(2,1))* ... 
(1-q(2,2))+(T1+T2-state(x7(ix),4)-state(x7(ix),5)+T3)*r(2)); 
end 
end 

num1 = 0; %stores the number of lines that have errors 
disp('Checking Transition Matrix for Integrity and Errors') 
for ix = 1 :numFeas 

if (abs(sum(P(ix,:))) -1 >= 0.0001) 
disp(['Error in Transition Matrix, line ' num2str(ix)]) 
numi = numi + 1; 
end 
end 

disp(['Number of lines containing errors:' num2str(num1)]) 

R = P(numAbs+1:numFeas, TnumAbs); 

Q = P(numAbs+1 :numFeas, numAbs+1:numFeas); 

I = eye(size(Q)); 

LQ = I - Q; 

l_Q_inv = inv(l_Q); 

I QJnv R = I QJnv * R; 

num2 = 0; %stores the number of lines that have errors 
dispC ') %insert an empty line on the screen 
disp('Checking Trans-to-Absorb Matrix for Integrity and Errors') 
for ix = 1 :numTrans 
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if (abs(sum(l_Q_inv_R(ix,:))) -1 >= 0.0001) 
disp(['Error in Trans-to-Absorb Matrix, line ' num2str(ix)]) 
num2 = num2 + 1; 
end 
end 

disp(['Number of lines containing errors:' num2str(num2)]) 

initialState = 0; 

for ix = numAbs+1:numFeas 

if( state(ix,2) + state(ix,3) + state(ix,4) + state(ix,5) == ... 

N1 + N2 + T1 +T2) 

initialState = ix - numAbs; 
break; 
end 
end 

expNumKilled_T1 = 0; 
for iy = 1 :numAbs 

expNumKilled_T1 = expNumKilled_T1 ... 

+ l_Q_inv_R(initialState, iy)*( T1 - state(iy,4)); 
end 

expNumKilled_T2 = 0; 
for iy = 1 :numAbs 

expNumKilled_T2 = expNumKilled_T2 ... 

+ l_Q_inv_R(initialState, iy) * ( T2 - state(iy,5)); 
end 

Ev = expNumKilled_T 1 * Tvalue(1) + expNumKilled_T2 * Tvalue(2); 


Time = 0; 

for k = 1 :numTrans 

Time = Time + l_Q_inv(initialState,k) /... 

(state(numAbs+k,2)*(lamda(1)+theta(1))+state(numAbs+k,3)*... 
(Iamda(2)+theta(2))); 
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A(j, col + 1) = Ev; 
A(j, col + 4) = Time; 


end 

save 129x17_Model_A_NOLH_output.xls A -ascii; 

dispC ') %insert an empty line on the screen 
if( num1 + num2 == 0 ) 
disp('Script successfully completed.') 
disp('Check workspace for more results.') 
else 

disp('There are flaws in the code. Debugging needed.') 
end 
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